{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[View the runnable example on GitHub](https://github.com/intel-analytics/BigDL/tree/main/python/nano/tutorial/notebook/inference/openvino/openvino_inference.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OpenVINO Inference using Nano API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use `OpenVINOModel` class in Nano to load an OpenVINO model and run inference. It only takes a few lines."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To run inference on OpenVINO model with Nano, the following dependencies need to be installed first:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for BigDL-Nano\n",
    "!pip install --pre --upgrade bigdl-nano # install the nightly-built version\n",
    "!source bigdl-nano-init\n",
    "\n",
    "# for OpenVINO\n",
    "!pip install openvino-dev "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 📝 **Note**\n",
    ">\n",
    "> We recommend to run the commands above, especially `source bigdl-nano-init` before jupyter kernel is started, or some of the optimizations may not take effect."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a [resnet18-xnor-binary-onnx-0001](https://docs.openvino.ai/latest/omz_models_model_resnet18_xnor_binary_onnx_0001.html) model pretrained on ImageNet dataset from the [Open Model Zoo](https://github.com/openvinotoolkit/open_model_zoo/) as an example. First, we download the model using [omz_downloader](https://docs.openvino.ai/latest/omz_tools_downloader.html):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!omz_downloader --name resnet18-xnor-binary-onnx-0001 -o ./model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To load the model and run inference, **the only change you need to make is to import BigDL-Nano OpenVINOModel, and pass the xml path of model to it**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bigdl.nano.openvino import OpenVINOModel\n",
    "\n",
    "ov_model = OpenVINOModel(\"model/intel/resnet18-xnor-binary-onnx-0001/FP16-INT1/resnet18-xnor-binary-onnx-0001.xml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then you could call `ov_model(x)` to run inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "x = np.random.randn(1, 3, 224, 224)\n",
    "# call ov_model directly to run inference\n",
    "y_hat = ov_model(x)\n",
    "predictions = y_hat.argmax(axis=1)\n",
    "print(predictions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 📚 **Related Readings**\n",
    "> \n",
    "> - [How to install BigDL-Nano](https://bigdl.readthedocs.io/en/latest/doc/Nano/Overview/install.html)\n",
    "> - [How to install BigDL-Nano in Google Colab](https://bigdl.readthedocs.io/en/latest/doc/Nano/Howto/Install/install_in_colab.html)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.13 ('nano-dev')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "bcba839278e72a05a80ee7542e90a94f8a25e052dc735aca32801a2a5657bf32"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
